Enhancement of image data

ABSTRACT

A technology is provided whereby correction may be carried out for both a person&#39;s face and other portions of an image, when performing color correction for a photographic image in which a person appears. During color correction of image data of a photographic image, the image data is analyzed, and a first region which is part of the photographic image and in which a person&#39;s face is present is determined. Then, on the basis of the portion corresponding to the first region of the image data, a first parameter relating to color is calculated. On the basis of part of the image data corresponding to a second region which is part of the photographic image but different from the first region, a second parameter relating to color is calculated. The color tone of the data is then corrected on the basis of the first and second parameters.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.11/709,634, filed on Feb. 21, 2007. The disclosure of this priorapplication is hereby incorporated by reference in its entirety for allpurposes.

BACKGROUND

1. Technical Field

This invention relates to a technology for enhancing color of imagedata.

2. Related Art

Technology for enhancing color of image data images on the basis of theimage data have been known for some time. For example, a region isdetermined in a photograph where a person's face appears, and colorenhancement of the image data is performed so that the color of theperson's face has a predetermined target chromaticity value.

However, since only the color of the person's face is taken intoconsideration when determining the content of color enhancement of imagedata, in some instances, color subsequent enhancement may appearunnatural in regions except for the person's face.

With the foregoing in view, an aspect of the invention is to providetechnology whereby when enhancement is being performed for image data ofa photographic image in which a human object appears, enhancement may becarried out appropriately for both the person's face and other portionsof the image.

The entire disclosure of Japanese patent application No. 2006-45039, ofSeiko Epson Corporation is hereby incorporated by reference into thisdocument.

SUMMARY

In order to address above problems, an image processing device forenhancing image data, as one aspect of the invention, has the followingportions. The image processing device includes: a first regiondetermining portion configured to analyze image data, and determine afirst region which is part of an image of the image data; a firstparameter calculating portion configured to calculate a first parameterbased on a portion of the image data which corresponds to the firstregion; a second parameter calculating portion configured to calculate asecond parameter based on a portion of the image data corresponding to asecond region which is part of the image and which differs from thefirst region; and an enhancing portion configured to enhance color toneof the image data based on the first and second parameters.

In enhancing color of image data, as another aspect of the invention,the following process may be performed. Image data is analyzed. Then afirst region which is part of an image of the image data is determined.A first parameter is calculated based on a portion of the image datawhich corresponds to the first region. A second parameter is calculatedbased on a portion of the image data corresponding to a second regionwhich is part of the image and which differs from the first region.Color tone of the image data is enhanced based on the first and secondparameters.

According to the aspect described above, parameters can be generatedrespectively while distinguishing between a first region and a secondregion different from the first region, and the image can be enhanced soas to reflect these individual parameters. Thus, when enhancement isperformed for an image, enhancement can be carried out appropriately fora plurality of objects.

The first region may be a region in which a human face is present. Insuch an aspect, the image data may be an image data of one or morephotographic images. The first and second parameters may be parametersrelating to color. The enhancement of the image may be performed bycorrecting color tone of the image data.

According to the aspect described above, parameters can be generatedrespectively while distinguishing between a first region in which ahuman face is present and a second region different from the firstregion, and the color tone of the image data can be enhanced so as toreflect these individual parameters. Thus, when enhancement is performedfor an image in which a human object appears, enhancement can be carriedout appropriately for both the person's face and other portions of theimage.

In the enhancing the image data, performing the following process ispreferable. A first target value representing a target value for thefirst parameter is prepared. A second target value representing a targetvalue for the second parameter is prepared. Then the image data isenhanced based on weighted average of (a) a difference of the firsttarget value and the first parameter and (b) a difference of the secondtarget value and the second parameter.

An advantage of this aspect is that, for example, respective targetvalues for the first and second regions can be established in advance tohave values corresponding to preferred color tone for the respectiveregions. By establishing target values for the first and second regionsin this way, enhancement can be performed in such a way as to adjustrespective color tone of the first and second regions to a predeterminedpreferred color tone.

In the enhancing the image data, it is preferable to calculate a weightfor the weighted average based on a difference of the first parameterand the second parameter. An advantage of this aspect is that, forexample, both in instances where the first and second regions havesimilar color tone and in instances where they differ appreciably,enhancement appropriate to the image can be carried out.

A weight for the weighted average may be calculated based on aproportion of the first region in the image. An advantage of this aspectis as follows. When face portions make up a large proportion of animage, such as when an image is a close-up of the face or when facesappear at numerous locations, enhancement can be performed, for example,so as to impart preferred color tone to the color of the face. When faceportions make up a small proportion of an image, enhancement can beperformed, for example, so as to impart preferred color tone to thecolor of the background.

The first parameter may be average brightness of the first region. Thesecond parameter may be average brightness of the second region. Anadvantage of this aspect is that enhancement can be carried out so as toapproximate color tone of preferred lightness both for a human face andfor other portions of an image.

In calculating the second parameter, it is preferable to determine, asthe second region, a region taken from a region except for the firstregion of the image and whose colors of neighboring pixels resemble eachother by at least a prescribed degree. An advantage of this aspect ishigher probability that enhancement will be carried out in such a way asto produce preferred color tone of an object which exists in thebackground, for example.

A process such as the following may be performed during enhancement ofimage data. One mode is selected from among a plurality of processingmodes. A third parameter is calculated based on the image data, when aprocessing mode of a first type is selected from among the plurality ofprocessing modes. The image data is enhanced based on the thirdparameter, when the processing mode of the first type is selected fromamong the plurality of processing modes. The image data is enhancedbased on the first and second parameters, when a processing mode of asecond type other than the first type is selected from among theplurality of processing modes.

The plurality of processing modes may includes: (1) a first processingmode of the first type for processing an image whose principal object isa human object; (2) a second processing mode of the first type forprocessing an image whose principal object is a landscape; and (3) athird processing mode of the second type for processing an image whoseprincipal objects are both a human object and a landscape.

An advantage of this aspect is that for an image having either a personor a landscape as the principal object, enhancement can be carried outappropriately for that principal object.

The third parameter may be a parameter relating to color.

In preferred practice, the third parameter may be determined based on aportion of the image data corresponding to a third region in the image,different from the first and second regions. For example, it would bepreferable to determine the third parameter based on the entire regionof the image.

In enhancing color of image data, as another aspect of the invention,the following process may be performed. Image data is analyzed todetermine first and second regions which are parts of an image of theimage data. Then the image data is enhanced based on both of aproportion of the first region in the image and a proportion of thesecond region in the image.

Various other possible aspects of the invention could include, forexample, an image processing method and image processing device; aprinting method and printing device; a computer program for achievingthe functions of such a method or device; a recording medium having sucha computer program recorded thereon; or a data signal containing such acomputer program and embodied in a carrier wave.

These and other objects, features, aspects, and advantages of thepresent invention will become more apparent from the following detaileddescription of the preferred embodiments with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the software arrangement of theprinting system of Embodiment 1;

FIG. 2 is a flowchart depicting the process of enhancement in the colortone adjustment module 103;

FIG. 3 shows a facial region A1 and a background subject region A2within an image A0 of preliminary image data PID;

FIG. 4 is a graph for specifying the weight a based on the differencedLa between the average luminance value La1 of pixels contained in thefacial region A1, and the average luminance value of the backgroundsubject region A2;

FIG. 5 is a tone curve illustrating a method of modifying luminance L*of pixels of image data PIDL;

FIG. 6 is a flowchart depicting the process of enhancement in Embodiment2;

FIG. 7 is a graph specifying the relationship of proportion Rf of thefacial region A1 to the weighting factor α of the facial region A1;

FIG. 8 is a flowchart of the enhancement process in Embodiment 4; and

FIG. 9 is a diagram depicting tone curves for the purpose of modifyingred, green, and blue tone values of the pixels of the preliminary imagedata PID.

DESCRIPTION OF EXEMPLARY EMBODIMENT A. Embodiment 1 A1. Overall DeviceConfiguration

FIG. 1 is a block diagram depicting the software configuration of theprinting system of Embodiment 1. On a computer 90, application software95 runs on a predetermined operating system. The operating systemincorporates a video driver 91 and a printer driver 96.

In response to an instruction by the user input from a mouse 130 or akeyboard 120, the application program 95 reads from a CD-R 140 originalimage data ORG having three color components, i.e. red (R), green (G),and blue (B). Then, in response to an instruction from the user, imageretouching or other such process is carried out on the original imagedata ORG. Through the agency of the video driver 91, the applicationprogram 95 displays the processed image on a CRT display 21. When theapplication program 95 receives a print command from the user, it issuesa print command to the printer driver 96 and outputs the processed imageto the printer driver 96 as preliminary image data PID. The preliminaryimage data PID is image data representing the colors of the pixels inthe sRGB color system. In the image data, each pixel has tone values(0-225) for the color components red (R), green (G), and blue (B).

The printer driver 96 receives the preliminary image data PID from theapplication program 95, and converts this data to print image data FNLprocessable by the printer 22 (here, a multilevel signal for six colors,namely, cyan, magenta, yellow, black, light cyan, and light magenta).

“Light cyan” is an ink color of the same hue as cyan, but with higherlightness than cyan. “Light magenta” is an ink color of the same hue asmagenta, but with higher lightness than.

In the example shown in FIG. 1, the printer driver 96 includes a colortone adjustment module 103, a resolution conversion module 97, a colorconversion module 98, a halftone module 99, and a sorting module 100.

The color tone adjustment module 103 converts the preliminary image dataPID received from the application program 95 into image data PIDr forenhancing the color tone. The image data PIDr is image data containingtone values (0-225) for the color components red (R), green (G), andblue (B) for each pixel, and has the same pixel count as the preliminaryimage data PID.

In the course of image processing in the printer driver 96, the user caninput commands via the mouse 130 or the keyboard 120 at prescribedtiming. A function module for prompting the user for prescribedinstructions through the display on the CRT 21 and for receiving inputvia the mouse 130 and keyboard 120 is depicted as UI module (userinterface module) 105 in FIG. 1.

The resolution conversion module 97 converts the resolution of the imagedata PIDr to the resolution at which printing will be carried out by theprinter 22, and generates image data MID1.

The color conversion module 98, while looking up in a three-dimensionallookup table 104 b, converts the image data MID1 to image data MID2. Theimage data MID2 is image data that represents color of pixels in termsof density of the ink colors used by the printer 22, i.e. cyan (C),magenta (M), yellow (Y), black (K), light cyan (LC), and light magenta(LM). The three-dimensional lookup table 104 b is a lookup table havingcombinations of tone values for red, green, and blue as input values andhaving combinations of tone values for cyan, magenta, yellow, black,light cyan, and light magenta as output values.

The halftone module 99 performs halftone processing on the image dataMID2 representing the density of each color for pixels in terms of tonevalues for each color, thereby converting the data to image data MID3representing the density of each color in terms of the dot on-off statefor each pixel (also referred to as “print data” or “dot data”).

The image data MID3 generated thereby is sorted by the sorting module100 in the order in which it is to be sent to the printer 22, and isoutput as the final print image data FNL.

The printer 22 comprises a mechanism for feeding paper P by means of afeed motor; a mechanism for reciprocating a carriage 31 in the directionMS perpendicular to the paper P feed direction SS, by means of acarriage motor; a print head 28 riding on the carriage 31, for ejectingink to form dots; P-ROM 42 for storing settings data of various kinds;and a CPU 41 for controlling the paper feed motor, the carriage motor,the print head 28, the P-ROM 42, and a control panel 32. The printer 22receives the print image data FNL, and in accordance with the printimage data FNL executes printing by forming dots on a printing mediumusing cyan (C), magenta (M), yellow (Y), black (K), light cyan (LC), andlight magenta (LM).

Herein, while “printing device” refers in the narrow sense to theprinter 22 only, but in the broader sense it represents the printingsystem as a whole including the computer 90 and the printer 22.

A2. Processing in Color Tone Adjustment Module

FIG. 2 is a flowchart depicting the process of enhancement in the colortone adjustment module 103. In Step S10, by operating the mouse 130and/or the keyboard 120, the user inputs a processing mode commandthrough the user interface screen which is displayed on the CRT 21. Theprocessing mode is selected from among a “portrait” mode for the purposeof processing a photographic image in which a person is the principalobject, a “landscape” mode for the purpose of processing a photographicimage in which a landscape is the principal object, and a“portrait+landscape” mode in which both a person and a landscape are theprincipal objects. In Step S10, the user interface module 105 of theprinter driver 96 displays the user interface screen on the CRT 21, aswell as receiving input from the user via the mouse 130 and/or thekeyboard 120.

FIG. 3 is shows a facial region A1 and a background subject region A2within an image A0 of preliminary image data PID. In the embodiment, thepreliminary image data PID sent from the application program 95 isphotographic image data that has been generated by a digital stillcamera. The image of the preliminary image data PID includes as objectsa person O1, sky O2, and sea O3. The photographic image of thepreliminary image data PID is a photographic image in which the personO1 and the background sky O2 and sea O3 are equally principal objects.In the case of processing an image of this kind, the user will selectthe “portrait+landscape” mode in Step S10.

In Step S20, the color tone adjustment module 103 determines whether theprocessing mode input in Step S10 was the “portrait+landscape” mode, orthe “portrait” mode or “landscape” mode. In the event that theprocessing mode input in Step S10 was the “portrait+landscape” mode, theprocess advances to Step S30. In the event that the processing modeinput in Step S10 was the “portrait” mode or “landscape” mode, theprocess advances to Step S90. Processes starting with Step S20 in theflowchart of FIG. 2 are executed by the color tone adjustment module103.

In Step S30, the color tone adjustment module 103 analyzes thepreliminary image data PID and determines the facial region A1. Thisdetermination of the facial region A1 may be accomplished, for example,by sequentially matching basic patterns of the human eyes and mouth withstandard patterns, for portion of the image of the preliminary imagedata PID. In the embodiment, a region of rectangular shape that includesthe eyes and the mouth is identified as the facial region A1 (see FIG.3). The functional part of the color tone adjustment module 103 foranalyzing the preliminary image data PID and determining the facialregion A1 is shown as facial region determining portion 103 a in FIG. 1.

In Step S40 of FIG. 2, the color tone adjustment module 103 determines abackground subject region A2 from a region of the image A0 except forthe facial region A1. The background subject region A2 may be determinedin the following manner, for example. Specifically, pixels contained ina region of the image A0 outside the facial region A1 are firstsequentially compared with neighboring pixels. The neighboring pixelsfor comparison for a pixel Po of interest are the neighboring pixel Pusituated above, and the neighboring pixel P1 situated at left (see FIG.3).

In the event that neighboring pixels are similar in color, the pixel Poof interest is classed into the same group as the comparison pixels.More specifically, in the event that neighboring pixels (e.g. pixel P1and pixel Po) have differences among their red, green, and blue tonevalues that are respectively smaller than certain prescribed values,pixel Po will be classed into the same group as the comparison pixel. Inthis way, pixels in areas of the image A0 except for the facial regionA1 are classified into one or more groups.

From among groups created in this way, the region occupied by the groupwith the most pixels is selected as the background subject region A2.The functional part of the color tone adjustment module 103 foranalyzing the preliminary image data PID and determining the backgroundsubject region A2 is shown as background region determining portion 103b in FIG. 1.

In Step S50, the preliminary image data PID in which colors of pixelsare represented by tone values in the RGB color system is converted toimage data PIDL which colors of pixels are represented by tone values inthe L*a*b color system.

In Step S60, an average luminance value La1 is calculated for the pixelscontained in the facial region A1. Specifically, the average value ofluminance L* is calculated for pixels of image data PIDL correspondingto pixels included in the facial region A1 of the preliminary image dataPID. The functional part of the color tone adjustment module 103 forcalculating the average luminance value La1 for the pixels contained inthe facial region A1 is shown as first parameter calculating portion 103c in FIG. 1.

In Step S70, an average luminance value La2 is calculated for the pixelscontained in the background subject region A2. In the same manner as inStep S60, the average value of luminance L* is calculated for pixels ofimage data PIDL corresponding to pixels included in the backgroundsubject region A2 of the preliminary image data PID. The functional partof the color tone adjustment module 103 for calculating the averageluminance value La2 for the pixels contained in the background subjectregion A2 is shown as second parameter calculating portion 103 d in FIG.1.

In Step S80, a luminance correction level dL is calculated from theaverage luminance value La1 for pixels contained in the facial region A1and the average luminance value La2 the pixels contained in thebackground subject region A2. The luminance correction level dL iscalculated using the following Eq. (1):

dL=α×(Lt1−La1)+(1−α)×(Lt2−La2)  (1)

Here, Lt1 is the target luminance of the facial region A1. Lt1 can beset to 70, for example. Lt2 is the target luminance of the backgroundsubject region A2. Lt2 can be set to 50, for example. These targetluminance values are held in a target value table 104 a by the printerdriver 96 (see FIG. 1). In Eq. (1), α is a weight appended to thedifference between the average luminance La1 and the target luminance ofthe facial region A1, and is a factor meeting the condition: (0<α≦1).Consequently, (1−α) is a weight appended to the difference between theaverage luminance La2 and the target luminance of the background subjectregion A2.

By prescribing luminance correction level dL in this way, the luminancecorrection level dL for an image can be determined while taking intoconsideration both deviance of the luminance of the facial region A1from the ideal luminance Lt1 of the facial region A1 (Lt1-La1), anddeviance of the luminance of the background subject region A2 from theideal luminance Lt2 of the background subject region A2 (Lt2-La2).

FIG. 4 is a graph for specifying the weight α based on the differencedLa between the average luminance value La1 of pixels contained in thefacial region A1, and the average luminance value of the backgroundsubject region A2. In the region in which the difference dLa of La1 andLa2 is ±20, α is a constant value α0=0.7. In the region in which dLa isgreater than 20, α increases in linear fashion in association withincreasing dLa. In the region in which dLa is less than −20, α decreasesin linear fashion in association with decreasing dLa. When dLa is 100, αis 1, and when dLa is −100, α is 0.

By specifying a in the manner illustrated in FIG. 4, when the facialregion A1 and the background subject region A2 have similar luminance(when −20<dLa<20), the weight appended to deviance of luminance of thefacial region A1 will be 0.7. In instances where a person and alandscape are both principal objects, a person viewing the photographicimage will be more sensitive to the brightness of the person's face thanto that of the background. Consequently, in cases where the facialregion A1 and the background subject region A2 have similar luminance,by determining a correction level through weighting that places moreemphasis on correcting deviance of the facial region A (α>0.5), it ispossible to correct luminance in such as way as to produce an image thatappears natural to the viewer.

Moreover, by specifying a in the manner illustrated in FIG. 4, ininstances where the facial region A1 has significantly higher luminancethan the background subject region A2 (dLa>20), weighting is carried outplacing more emphasis on correcting deviance of the facial region A1 ascompared to when the two are similar (i.e. when −20<dLa<20). Bydetermining the weighting level in this way, it is possible to preventcircumstances in which correction results in unnatural whiteness of theperson's face.

Furthermore by specifying α in the manner illustrated in FIG. 4, ininstances where the facial region A1 has significantly lower luminancethan the background subject region A2 (dLa<−20), weighting is carriedout placing more emphasis on correcting deviance of the backgroundsubject region A2 as compared to when the two are similar (i.e. when−20<dLa<20). By determining the weighting level in this way, it ispossible to prevent circumstances in which correction results inunnatural whiteness of the background.

Where the difference in luminance of the facial region A1 and of thebackground subject region A2 are about the same, weighting is carriedout placing more emphasis on the facial region. For example, in theevent that dLa=50 and the facial region A1 is brighter, the weight a ofthe facial region A1 is 0.8125. On the other hand, in the event thatdLa=−50 and the background subject region A2 is brighter, α is 0.4375and the weight (1−α) of the background subject region A2 is 0.5625. Asnoted earlier, people are more sensitive to the brightness of a facethan to the brightness of the background. Consequently, by performingweighting in this manner, it is possible to correct brightness so as toafford an image of natural appearance.

FIG. 5 shows a tone curve CL that illustrates a method of modifyingluminance L* of pixels of image data PIDL. Luminance input values Li areplotted on the horizontal axis and luminance output values Lo on thevertical axis. The tone curve of FIG. 5 is a tone curve which, at aluminance input value of Lir, increases luminance by dL. The value ofLir could be, for example, 50, which represents the median value of theluminance tone values 1-100. The tone curve of FIG. 5 is a quadratictone curve that replaces a luminance input value of 0 with a luminanceoutput value of 0, and a luminance input value of 100 with a luminanceoutput value of 100. In FIG. 5, the output value corresponding to theinput value Lir is denoted as Lor. In actual practice, the tone curvedescribed herein is provided as a table that includes input values andoutput values that are discrete values.

In Step S120, the color tone adjustment module 103 generates the tonecurve of FIG. 5, in accordance with the dL calculated in Step S80. Theluminance of the pixels of the image data PIDL represented by tonevalues in the L*a*b* color system are then modified in accordance withthe tone curve of FIG. 5. Hereinafter this modified image data shall bedenoted as image data PIDLr.

In Step S130, the image data PIDLr created in this way is converted toimage data PIDr represented by tone values (0-255) in the RGB colorsystem (see FIG. 1). The functional part of the color tone adjustmentmodule 103 that performs the functions of generating the tone curve,image conversion to modify color tone, and creation of the PIDr data inthe above manner is shown as an enhancing portion 103 f in FIG. 1.

On the other hand, if in Step S20 of FIG. 2 the processing mode input inStep S10 is the “portrait” mode or “landscape” mode, the processadvances to Step S90.

In Step S90, in the same manner as in Step S50, the preliminary imagedata PID representing pixel colors as tone values in the RGB colorsystem is converted to image data PIDL representing pixel colors as tonevalues in the L*a*b* color system.

In Step S100, an average luminance value La3 for luminance L* of allpixels of the image data PIDL is calculated. The function of calculatingthe average luminance value La3 for luminance L* of all pixels of theimage data PIDL is performed by a third parameter calculation portion103 e constituting a functional part of the color tone adjustment module103 (see FIG. 1).

Subsequently, the luminance modification level dL is calculated inaccordance with the following Eq. (2).

dL=Lt3−La3  (2)

Here, Lt3 is target luminance. Where “portrait” mode has been selectedin Step S10, Lt3 will be replaced with the target luminance Lt1 of thefacial region A1. On the other hand, where “landscape” mode has beenselected in Step S10, Lt3 will be replaced with Lt2. These targetluminance values Lt1, Lt2 are held in the target value table 104 a bythe printer driver 96 (see FIG. 1).

Subsequently, the processes in Steps S120 and S130 are carried out inthe manner described previously. The functions of Steps S110-S130 areperformed by an enhancing portion 103 f constituting a functional partof the color tone adjustment module 103.

As discussed previously, in Embodiment 1, in the “portrait+landscape”mode process, average values for luminance La1, La2 are calculatedrespectively for the facial region A1 in the image data and thebackground subject region A2 different from the facial region A1; and onthe basis of these average values, the luminance of the preliminaryimage data PID is adjusted to created image data PIDr. Thus, there islittle likelihood of unnatural brightness of the background such as canoccur where a luminance adjustment level is determined on the basis offacial region data exclusively. Moreover, there is little likelihood ofunnatural brightness of a person's face such as can occur where aluminance adjustment level is determined on the basis of background dataexclusively. That is, color tone can be adjusted in such a way that bothpeople and background will have brightness levels that appear natural tothe user.

In Embodiment 1, during processing in “portrait” mode and in “landscape”mode, the average value of luminance La3 for the image as a whole iscalculated, and on the basis of this average value the luminance of thepreliminary image data PID is adjusted to created image data PIDr. Thus,the brightness of the preliminary image data PID image as a whole can beadjusted to brightness appropriate to a portrait photograph and alandscape photograph.

B. Embodiment 2

In Embodiment 2, a photographic image “scene” is determined on the basisof regions except for the facial region A1, and the luminance targetvalues Lt1, Lt2 are determined on the basis of the determined “scene.”Other processes and device configurations of Embodiment 2 are the sameas those of Embodiment 1.

FIG. 6 is a flowchart depicting the process of enhancement in Embodiment2. The processes of the flowchart of FIG. 6 are the same as those of theflowchart of FIG. 2, except for Steps S43, S45, S93, and S95. StepsS50-S70, which have the same process content as in the flowchart of FIG.2, have been omitted from the illustration in the flowchart of FIG. 6.

In Embodiment 2, after Step S40, the color tone adjustment module 103 inStep S43 determines a “scene” of the preliminary image data PID image,on the basis of the preliminary image data PID. The “scene” is selected,for example, from among a plurality of alternatives such as “clear sky,”“sunset sky,” “trees,” and so on. “Scenes” can be determined based onaverage color of all pixels of the preliminary image data PID.Specifically, “scenes” can be determined on the basis of the averagevalue of red tone values, the average value of green tone values, andthe average value of blue tone values for all pixels in the preliminaryimage data PID. In the event that the preliminary image data PID storesExif (Exchangeable Image File Format) data, “scenes” can be determinedon the basis of the Exif data.

In Step S45, on the basis of the scene determined in Step S43, theluminance target value Lt1 of the facial region A1 and the luminancetarget value Lt2 of the background subject region A2 are determined. Forexample, where the “scene” is “sunset sky,” Lt2 will be set to 30. Wherethe “scene” is “clear sky,” Lt2 will be set to 50. Similarly, theluminance target value Lt1 of the facial region A1 will be determineddepending on the scene. Luminance target values Lt1, Lt2 for these“scenes” are already stored in the target value table 104 a by theprinter driver 96 (see FIG. 1).

After Step S45, processing is carried out analogously to the processingbeginning with Step S50 of Embodiment 1 (see FIG. 2).

In the meantime, after Step S90, Step S93 is executed. The processcontent of Step S93 is the same as in Step S43. Specifically, a “scene”is determined on the basis of the preliminary image data PID. Then, inStep S95, a luminance target value Lt3 is determined depending on the“scene” determined in Step S93 and the processing mode input in StepS10. Luminance target values Lt3 for these “scenes” and processing modesare already stored in the target value table 104 a by the printer driver96.

After Step S95, processing starting with Step S100 is carried outanalogously to the flowchart of FIG. 2.

According to Embodiment 2, target luminance depending on a particularscene can be established for each set of image data. Thus, it ispossible to prevent a situation where an image appears unnaturallybright as a result of correction, even where the scene is a sunsetscene. That is, enhancement can be carried out in a manner appropriateto the content of the image data.

C. Embodiment 3

In Embodiment 1, the weighting factor α for determining the luminancecorrection level dL was determined based on the difference dLa betweenthe average value of luminance La1 of pixels in the facial region A1 andthe average value of luminance La2 of pixels in the background subjectregion A2 (see FIG. 4). In Embodiment 3, the weighting factor isdetermined based on the proportion Rf of the entire image A0 occupied bythe facial region A1. The processes of Steps S90-S110 of FIG. 2 are notexecuted in Embodiment 3, but in other respects Embodiment 3 is the sameas Embodiment 1.

FIG. 7 is a graph Cw specifying the relationship of proportion Rf of thefacial region A1 to the weighting factor α of the facial region A1. Whenthe proportion Rf of the facial region A1 is 0, the weighting factor αis 0 as well, and when the proportion Rf of the facial region A1 is100%, the weighting factor α is 1. α increases along a convex curve inassociation with increasing Rf. The proportion Rf of the facial regionA1 is derived by dividing the pixel count of the facial region A1 by thetotal pixel count of the image.

In Embodiment 3, the larger the proportion of the image A0 occupied bythe facial region A1 is, the greater the extent to which weighting willemphasize correcting deviance of the facial region A in thedetermination of the correction level dL (see Eq. (1)). The larger theproportion of the image A0 occupied by the background subject region A2is, the greater the extent to which weighting will emphasize correctingdeviance of the background subject region A2 in the determination of thecorrection level dL. In other words, luminance correction will becarried out weighting with greater emphasis on the region that occupiesa large proportion of the image and that will be quite noticeable to theuser. It is therefore possible to produce images that will be assessedhighly by the user.

The weighting factor α increases along a convex curve in associationwith increasing Rf. For an area of the same given size in an image, thehuman eye will place greater emphasis on the color tone of a human facethan on the color tone of the background. Consequently, by performingluminance correction while determining the weighting factor along acurve like that of FIG. 7, it is possible to produce images that will beassessed highly by the user.

D. Embodiment 4

In Embodiments 1-3, luminance correction was performed for thepreliminary image data PID, and then image data PIDr was created (seeFIG. 1 and FIG. 5). In Embodiment 4, image data PIDr is created throughmodification of red, green, and blue tone values of the pixels of thepreliminary image data PID on the basis of a tone curve. In otherrespects, Embodiment 4 is the same as Embodiment 2.

FIG. 8 is a flowchart of the enhancement process in Embodiment 4. Theprocesses of Steps S10-S43, S90, and S93 of the flowchart of FIG. 8 arethe same as in the flowchart of FIG. 6.

In Step S47, on the basis of the scene determined in Step S43, red,green, and blue target tone values Vrt1, Vgt1, Vbt1 of the facial regionA1 and red, green, and blue target tone values Vrt2, Vgt2, Vbt2 of thebackground subject region A2 are determined. These “scene”-dependent RGBtarget tone values are already stored in the target value table 104 a bythe printer driver 96 (see FIG. 1).

In Embodiment 4, conversion from the preliminary image data PID toL*a*b* color space image data PIDL does not take place (see Step S50 ofFIG. 2). As a result, reverse conversion from image data PIDL to RGBcolor system image data PIDr does not take place either (see Step S130of FIG. 2).

In Step S65, average values Vra1, Vga1, Vba1 of the red, green, and bluetone values of the facial region A1 are calculated. In Step S75, averagevalues Vra2, Vga2, Vba2 of the red, green, and blue tone values of thebackground subject region A2 are calculated.

In Step S85, correction levels dR, dG, dB of the red, green, and bluetone values are calculated with the following equations. α4 is aweighting factor relating to correction of RGB tone values of the facialregion A1.

dR=α4×(Vrt1−Vra1)+(1−α4)×(Vrt2−Vra2)  (3)

dG=α4×(Vgt1−Vga1)+(1−α4)×(Vgt2−Vga2)  (4)

dB=α4×(Vbt1−Vba1)+(1−α4)×(Vbt2−Vba2)  (5)

FIG. 9 shows tone curves for the purpose of modifying red, green, andblue tone values of the pixels of the preliminary image data PID. Thehorizontal axis in FIG. 9 indicates red, green, and blue input tonevalues Vi (0-255), while the vertical axis indicates red, green, andblue output tone values Vo (0-255). The red tone curve Cr is a tonecurve that increases the input tone value Virgb by dR. The green tonecurve Cg is a tone curve that increases the input tone value Virgb bydG. The blue tone curve Cb is a tone curve that increases the input tonevalue Virgb by dB. In the example of FIG. 9, dR and dG are positivevalues, and dB is a negative value. The input tone value Virgb can be128, which is the medial value between 0 and 255.

The tone curves Cr, Cg, Cb are each a quadratic tone curve thatsubstitutes an output tone value of 0 for an input tone value of 0, andan output tone value of 255 for an input tone value of 255. In FIG. 9,Vor denotes the red output tone value corresponding to the input tonevalue Virgb. Vog denotes the green output tone value corresponding tothe input tone value Virgb. Vob denotes the blue output tone valuecorresponding to the input tone value Virgb.

In Step S125 of FIG. 8, tone curves Cr, Cg, Cb like those depicted inFIG. 9 are generated based on the correction level dR, dG, dB, and imageconversion is carried out on the basis of the tone curves Cr, Cg, Cb. Asa result, image data PIDr is created from the preliminary image data PID(see FIG. 1).

In the meantime, in Step S97, red, green, and blue target tone valuesVrt3, Vgt3, Vbt3 are determined depending on the “scene” determined inStep S93 and on the processing mode input in Step S10. These RGB targettone values dependent on the “scene” and the processing mode are alreadystored in the target value table 104 a by the printer driver 96.

In Step S105, average values Vra2, VGa3, Vba3 of the red, green, andblue tone values of all pixels of the preliminary image data PID arecalculated. In Step S115, red, green, and blue modification levels dR,dG, dB are calculated according to the following Equations (6) to (8).

dR=Vrt3−Vra3  (6)

dG=Vgt3−Vgra3  (7)

dB=Vbt3−Vba3  (8)

Subsequently, in Step S125, the tone curves Cr, Cg, Cb (see FIG. 9) aregenerated, and image conversion is carried out on the basis of thesetone curves Cr, Cg, Cb.

According to Embodiment 4, it is possible to adjust not only luminance,but also saturation and hue for an image of image data, so that bothpeople and background in the image will have brightness that appearsnatural to the user.

E. Modified Examples

The invention is not limited to the embodiments and aspects describedherein above, and may be reduced to practice in various forms withoutdeparting from the spirit thereof. Modified examples such as thefollowing are possible, for example.

E1. Modified Example 1

In the preceding embodiments, there was a single facial region A1 (seeFIG. 3). However, a plurality of facial regions could be identifiedwithin an image. Specifically, one or more facial regions could beidentified on the basis of pattern matching or pixel color.Additionally, when calculating average values of luminance and of tonevalues of color components, it is possible for average values to becalculated based on all pixels in this plurality of facial regions.Alternatively, it is possible for average values to be calculated basedon a prescribed number of facial regions taken, in order from thelargest, from among a plurality of facial regions. Additionally, whererankings have been assigned to a plurality of facial regions in orderfrom the largest, it is possible for average values to be calculatedbased on facial regions with rankings above a certain prescribedproportion.

E2. Modified Example 2

In the preceding embodiments, the background subject region A2constitutes a region which is part of the region except for the facialregion A1 (see FIG. 3). However, the background subject region A2 couldinstead constitute the entire region of the image A0 except for thefacial region A1. Alternatively, the background subject region A2 couldbe established so as to include the region except for the facial regionA1, as well as part of the facial region A1. The background subjectregion A2 could also be established so as to include the region exceptfor the facial region A1, as well as the entire facial region A1.

In the preceding embodiments, in the event that the color of a givenpixel is similar to the color of an adjacent pixel situated above or tothe left side, the pixels are classified into the same group (see FIG.3). However, comparisons among pixels could instead be carried outbetween a given pixel and an adjacent pixel situated to the right side,or between a given pixel and an adjacent pixel situated below it.

Groups of pixels of the region except for the facial region A1 couldalso be determined by some method other than comparing color amongadjacent pixels. For example, pixels of color similar, by more than acertain extent, to a prescribed standard color such as clear skystandard color, sunset sky standard color, or tree green, could beclassified into the same group.

Furthermore, in the preceding embodiments, a single group with thehighest pixel count among such groups was designated as the backgroundsubject region A2. However, the background subject region A2 could bedetermined by some other method. For example, the background subjectregion A2 could be the background subject region A2 for a prescribednumber of groups from among the largest of the plurality of groups.Additionally, where rankings have been assigned to a plurality of groupsin order from the largest, only groups of ranking above a certainprescribed proportion could be designated as the background subjectregion A2.

In enhancing the image, the first region may be decided in any waysother than that of the preceding embodiments. The second region may alsobe decided in any ways other than that of the preceding embodiments. Thesecond region may be a region different from the first region. The firstand second regions may share a part of the region. The image enhancementmay be performed based on one or more parameters calculated based on thefirst and second region.

E3. Modified Example 3

In Embodiment 1, when the difference between luminance of the facialregion A1 and the background subject region A2 is within a given rangethat includes 0, the weighting factor a0 of the facial region A1 was 0.7(see FIG. 4). However, some other value may be used as the weight a0 ofthe facial region A1 when the difference between luminance of the facialregion A1 and the background subject region A2 is within a given range.However, in preferred practice a0 will be a value greater than 0.5. Byspecifying the weighting factor a0 of the facial region A1 in this way,when the facial region A1 and the background subject region A2 aresimilar in luminance, correction can be carried out with emphasis oncorrecting color of the facial region as opposed to the background. Inpreferred practice, when the difference between luminance of the facialregion A1 and the background subject region A2 is within a given range,the weighting factor a0 of the facial region A1 will preferably be0.6-0.8, more preferably 0.65-0.75.

The graph specifying the weighting factor α0 of the facial region A1 canbe constituted as a curve. In preferred practice, however, the slope ofα within a given range that includes a difference dLa of 0 betweenluminance of the facial region A1 and the background subject region A2will be smaller than the slope of a where dLa is outside of that therange.

Additionally, the weighting factor α0 of the facial region A1 within agiven range that includes a difference dLa of 0 between luminance of thefacial region A1 and the background subject region A2 can also bedetermined on the basis of the proportion Rf of the image occupied bythe facial region A1. The weighting factor α0 of the facial region A1can also be specified on the basis of both the difference in luminanceof the facial region A1 and or the background subject region A2, and theproportion Rf of the image occupied by the facial region A1 (see FIG. 4and FIG. 7).

E4. Modified Example 4

In Embodiment 1, the tone curve for carrying out correction of luminancewas a tone curve passing through the three points (0, 0), (Lir, Lor),and (100, 100) (see FIG. 5). The input tone value Lir used as thestandard when generating the tone curve was L*=50. However, some othervalue could be used as the input tone value Lir for the standard whengenerating the tone curve. For example, a value equivalent to idealluminance of a person's face (e.g. L*=70) could be used. The input tonevalue Lir used as the standard when generating the tone curve can alsobe specified on the basis of “scene” (see Step S10 of FIG. 2) orprocessing mode (see Steps S43, S93 of FIG. 6).

Also, in Embodiment 1, the tone curve for luminance was a quadraticcurve passing through three points (see FIG. 5). However, the tone curvecould instead be a cubic curve, a quartic curve, a spline curve, aBezier curve, or some other curve.

Similarly, for the red, green, and blue tone curves Cr, Cg, Cb ofEmbodiment 4 as well, input tone value Virgb used as the standard canassume various values, as with the luminance tone curve. Also, the inputtone value used as the standard can have mutually independent values forred, green, and blue. The red, green, and blue tone curves Cr, Cg, Cbcould also be cubic curves, quartic curves, spline curves, Beziercurves, or some other curve, instead of quadratic curves.

E5. Modified Example 5

In Embodiment 2, the “scene” was specified on the basis of the image asa whole (see Steps S43, S93 of FIG. 6). However, scenes could bespecified by some other method. For example, “scene” could be specifiedon the basis of a region except for the facial region A1, from theentire image A0.

E6. Modified Example 6

In the preceding embodiments, the printing system was constituted as asystem including the computer 90, the CRT display 21, the mouse 130and/or keyboard 120, and the printer 22. However, the printing systemwhich is one aspect of the present invention could also be constitutedas an integrated printer having a card hold able to read a memory withimage data stored thereon, a display able to display a user interfacescreen, buttons allowing the user to input commands, and the modules ofthe printer driver 96 in Embodiment 1.

E7. Modified Example 7

In the preceding embodiments, some of the arrangements achieved throughhardware could be replaced by software, and conversely some of thearrangements achieved through software could be replaced by hardware.For example, some of the functions of the printer driver 96 (see FIG. 1)could be executed by an application program 95, application programs orthe CPU 41 of the printer 22.

A computer program for realizing such functions could be provided in aform recorded on a flexible disk, CR-ROM, or other suchcomputer-readable recording medium. The host computer could read thecomputer program from the recording medium and transfer it to aninternal storage device or an external storage device. Alternatively,the computer program may be provided to the host computer from a programproviding device via a communications pathway. When realizing thefunctions of a computer program, the computer program stored in aninternal storage device is executed by the microprocessor of the hostcomputer. Alternatively, the computer program recorded on the recordingmedium can be executed directly by the host computer.

“Computer” herein refers to a concept that includes hardware devices andan operating system, and means that the hardware devices operate underthe control of the operating system. The computer program accomplishesthe functions of the parts described above on such a host computer. Someof the aforementioned functions can be realized by the operating system,rather than by an application program.

In this invention, “computer-readable recording medium” is not limitedto flexible disks, CR-ROM, or other portable recording media, but caninclude computer internal storage devices such various kinds of RAM andROM, as well as hard disks and other external storage devices fixed tothe computer.

The program product may be realized as many aspects. For example:

(i) Computer readable medium, for example the flexible disks, theoptical disk, or the semiconductor memories;(ii) Data signals, which comprise a computer program and are embodiedinside a carrier wave;(iii) Computer including the computer readable medium, for example themagnetic disks or the semiconductor memories; and(iv) Computer temporally storing the computer program in the memorythrough the data transferring means.

While the invention has been described with reference to preferredexemplary embodiments thereof, it is to be understood that the inventionis not limited to the disclosed embodiments or constructions. On thecontrary, the invention is intended to cover various modifications andequivalent arrangements. In addition, while the various elements of thedisclosed invention are shown in various combinations andconfigurations, which are exemplary, other combinations andconfigurations, including more less or only a single element, are alsowithin the spirit and scope of the invention.

1. An image processing apparatus comprising: a receiving portion thatreceives from a user an instruction of one of image processing modesincluding a “human and landscape” mode; and an enhancement portion thatperforms an image enhancement on image data according to the receivedinstruction, wherein the enhancement portion is able to perform imageenhancement based on pixel values of one or more faces in the image dataand pixel values of one or more objects other than the one or more facesin the image data, the one or more faces being not all but part of facesincluded in the image data.
 2. An image processing apparatus of claim 1further comprising: a specifying portion that is able to specify facesin the image data, wherein the image processing modes further include a“human” mode, wherein in a case where the received instruction is aninstruction of the “human and landscape” mode, the specifying portionspecifies faces in the image data, and in a case where the receivedinstruction is an instruction of the “human” mode, the specifyingportion does not specify faces in the image data.
 3. An image processingmethod comprising: receiving from a user an instruction of one of imageprocessing modes including a “human and landscape” mode; and performingan image enhancement on image data according to the receivedinstruction, wherein the image enhancement is performed based on pixelvalues of one or more faces in the image data and pixel values of one ormore objects other than the one or more faces in the image data, the oneor more faces being not at all but part of faces included in the imagedata.
 4. A non-transitory computer readable medium storing a computerprogram, the computer program comprising: a portion for receiving from auser an instruction of one of image processing modes including a “humanand landscape” mode; and a portion for performing an image enhancementon image data according to the received instruction, wherein the imageenhancement is performed based on pixel values of one or more faces inthe image data and pixel values of one or more objects other than theone or more faces in the image data, the one or more faces being not allbut part of faces included in the image data, wherein the imageprocessing modes further include a “human” mode, wherein the computerprogram further comprises a portion for specifying faces in the imagedata, wherein in a case where the received instruction is an instructionof the “human and landscape” mode, faces in the image data are specifiedthrough the portion for specifying faces, and in a case where thereceived instruction is an instruction of the “human” mode, faces in theimage data are not specified by the portion for specifying faces.